Duality Notes 
18.310, Fall 2005, Prof. Peter Shor 

One of the most important aspects of linear programming is the duahty theorem. 
Let's consider a Unear program in the standard form we talked about last time. 

max.y^ vj.Xk subject to 
k 

X! "^jkXk < bj Vj and 
k 

Xk >Q yk 

Now, how might we come up with an upper bound for the objective function 

Recall the example we did at the end of last class, where we took a sum of some 
multiple of the inequalities and found that this gave us an upper bound on the 
objective function? We'll use the same trick, in greater abstraction. 

So what we'll try to do is to take all these inequalities 

Ajk^k ^ bj, 

k 

multiply each of them by some value pj, and add all the equations up. We want to 
choose these values of yj so that this is a bound on Xk- What happens when we 
sum these inequalities? We get 

Y^yiJl^jk^k <J2bjyj. 

j k j 

Wc need that all the yj > 0, because if we multiplied by a negative yj, we would 
reverse the sign of the inequality. 

However, if all the yj are positive, then the above equation must hold for any 
feasible point {xi,X2, ■ ■ ■ ,Xn}, since a feasible point satisfies all the inequalities. 
How can we make sure that the resulting sum is a bound on J^'^k^k'^ We need to 
make sure that 

^VkXk < J^Jlyj'^jkXk 
k 3 k 

for all feasible points x^. One way to ensure this is to make sure that the coefficient 
on Xk on the left is less than the coefficient on Xk on the right. Thus, we would need 

Vk<^y3-^3k "^k. 

3 
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Note that here we are using the fact that > 0. 

Any set of yj satisfying the above equations gives us an upper bound J2j ^jUj on 
the objective function. How do we get the best bound on the objective function? 
We need to minimize J2j ^jUj- 

When we minimize J2jbjyj, we get another linear program, although this one 
isn't in our standard form. We have the Unear program 

min ^ bjyj subject to 

3 

X] ^jkVj > Vk Vfc and 
j 

Vj > Vj. 

What have we done? The right-hand side of the constraints, variables bj, have 
switched places with the constants in the objective, variables v^- We've effec- 
tively transposed the matrix Ajk, so instead of J2k ^jk^k we have J2j -^jkUj- We've 
swapped min for max, and < for >. 

Now, we've got a new linear program, which we call the dual. We call the 
original linear program the primal. The dual of the dual will be the primal. We 
have seen that if we have two feasible sets of variables and yj (for the primal and 
dual, respectively), the objective function of the dual is always at least the objective 
function of the primal. This shows that the optimal value of the dual is always at 
least the optimal value of the primal. 

Now, a truly amazing fact about linear programming, and the source of a lot of 
its effectiveness, is that these two values are equal. This is known as the duality 
theorem of linear programming. We will prove it in a little bit. The theorem 
actually says a bit more, about infeasible and unbounded linear programs. We won't 
prove this in class, but it's not hard to generalize the proof to handle these cases. 

Theorem: (The duality theorem for linear programming) 

If both the primal and the dual are feasible and unbounded, the optimal value of 
the primal is equal to the optimal value of the dual. The primal is infeasible if the 
dual is unbounded. The dual is infeasible if the primal is unbounded. 

It is possible for both the dual and the primal to be infeasible. One can get this 
situation by combining a linear progrma with an infeasible primal and unbounded 
dual by one with an unbounded primal and infeasible dual. 

There is a recipe for taking a linear program (whether or not it's in standard form) 
and finding it's dual. Inequalities turn into variables yj with the constraint yj > 0. 
Equations turn into variables yj which can be either positive or negative, and so 
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on. However, the recipe is complicated enough that if you memorize it, you're likely 
to forget it or remember it wrong when you have to use it, especially if you try to 
memorize the recipe for general linear programs and not just standard form. The 
best way of remembering how to find the dual is remembering the proof above that 
the dual optimum is at least the primal optimum, and reproducing it whenever you 
need it. 

We now want to prove the duality theorem. That is, we want to show that 
the optimal value of the primal is equal to the optimal value of the dual. We've 
shown one inequality already (this was the easy one), so now we need to show the 
other inequality. We will be able to do this by looking at the final tableau in the 
simplex algorithm, and show that the objective function it gives is not only a feasible 
solution for the primal but also a feasible solution for the dual. 

So let's recall the simplex algorithm. We started by taking all the inequalities 
that weren't of the form > and adding a slack variable Sj to them, so that we 
get inequalities 

Sj > Vj 

and equalities of the form 

k 

We then let all these equalities be a row in a tableau, and put the objective function 
on the bottom row, and performed row operations on the tableau. We stopped (and 
claimed we had the optimum value of the objective function) when the bottom row 
had all non-positive entries. Suppose that at the end of the algorithm, the last row 
is 

Wi,W2,W3,...,Wn,ri,r2,...,rm\ -F 

where Wk is in the column whose variable is x^, and is in the column whose 
variable is s^. Assuming that the simplex algorithm found an optimum, all entries 
in the last row will be non-positive, and there will be at least m zeros. 

Now, we know this last row is obtained by taking the original objective function 

■yi,'y2,'y3,---,'yn,0,0, ...,0|0 

and adding some linear combination of the rows to it. A typical row is 

Aj^i,Aj^2Aj,z, . . . Aj^n, 0, . . . , 0, 1, 0, . . . , 1 

where the 1 is in the column belonging to Sj . Now, suppose we take the last row to 
be the original last row minus yj times the j'th row. This means that 

Wk = Vk-^ VjAj^k 

3 
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and 

= -Vj- 

Since Wh < 0, we see that 

Vk<^ Aj,kyj VA;. 

3 

Since rj < 0, we see that 

yj > 0. 

And finally, from the last column, we have that the optimum value of the objective 
function, F satisfies 

j 

We have shown that the optimal value of the primal is equal to the optimal value 
of the dual. 

There's more information we can learn from the solution of the dual. If an 
equation in the primal LP is satisfied with strict inequality, then the corresponding 
dual variable yj must be in the optimal dual solution, because otherwise when we 
multiply this equation by yj, we introduct an inequality, and the primal and dual 
optima would not be equal. Similarly, if a variable in the primal LP is non-zero in 
the optimum solution, the corresponding equation in the dual LP must be satisfied 
with equality in the optimal solution to the dual. 

Often, if there is some intuitive interpretation of the linear program (for example, 
for maximum flow in a graph), there will also be some intuitive interpretation of 
the dual linear program (in this case for minimum cut in the graph). The equality 
of these two linear programs then may correspond to a combinatorial theorem. 
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